IN-DOMAIN DATA

Learning a Grammar Inducer from Massive Uncurated Instructional Videos (2022.emnlp-main)

Challenge:	Existing supervised classification models are insensitive to class names, but are no longer effective in open-domain tasks where the taxonomy is unbounded.
Approach:	They propose a topic classification system that accepts user-defined taxonomy in real time . they train a pretrained language model on a new Wikipedia dataset and train it on Wikipedia .
Outcome:	The proposed system improves over existing zero-shot models and performs competitively with weakly-supervised models trained on in-domain data.

Copied to clipboard

Songyang Zhang, Linfeng Song, Lifeng Jin, Haitao Mi, Kun Xu, Dong Yu, Jiebo Luo

Visual Data in NLP

Challenge:	aims to find more accurate syntactic grammars for accompanying text using video data.
Approach:	They build a video-aided grammar induction model that can learn video-span correlation without manual features.
Outcome:	The proposed model can learn video-span correlation without manual features adopted by previous systems.

Building a Corpus from Handwritten Picture Postcards: Transcription, Annotation and Part-of-Speech Tagging (L18-1)

Copied to clipboard

Kyoko Sugisaki, Nicolas Wiedmer, Heiko Hausendorf

Syntactic Text Processing

Challenge:	In this paper, we describe the processes and challenges of digitalisation, manual transcription, and manual annotation of over 11,000 postcards.
Approach:	They describe the processes and challenges of digitalisation, manual transcription, and manual annotation of over 11,000 postcards written in German and Swiss German.
Outcome:	The proposed system outperforms state-of-the-art taggers in the evaluation of the 'picture postcard corpus' containing over 11,000 handwritten postcards .

DiaSynth: Synthetic Dialogue Generation Framework for Low Resource Dialogue Applications (2025.findings-naacl)

Copied to clipboard

Sathya Krishnan Suresh, Wu Mengjun, Tushar Pranav, EngSiong Chng

Text Generation

Challenge:	Existing research is limited by general or niche datasets that lack sufficient scale for training dialogue systems.
Approach:	They propose a synthetic dialogue generation framework that uses Large Language Models and Chain of Thought reasoning to generate dynamic, domain-specific dialogues with simulated personas and diverse conversational features.
Outcome:	The proposed framework outperforms existing frameworks on dialogue summarization and quality increases as the size of the LLM increases from 3B to 8B.

MTNT: A Testbed for Machine Translation of Noisy Text (D18-1)

Copied to clipboard

Paul Michel, Graham Neubig

Machine Translation

Challenge:	Noisy input text can cause disastrous mistranslations in most modern machine translation systems.
Approach:	They propose a benchmark dataset for Machine Translation of Noisy Text (MTNT) they use reddit comments and professionally sourced translations to examine noise types.
Outcome:	The proposed dataset can provide an attractive testbed for noise-robust machine translation systems.

TwistList: Resources and Baselines for Tongue Twister Generation (2023.acl-short)

Copied to clipboard

Tyler Loakman, Chen Tang, Chenghua Lin

Syntactic Text Processing

Challenge:	Previous work in phonetically-grounded language generation has focused on domains such as lyrics and poetry.
Approach:	They propose to use TwistList to generate phonetically constrained tongue twisters, a large annotated dataset consisting of 2.1K+ human-authored examples.
Outcome:	The proposed models perform better than pre-trained models with limited training and data and no explicit phonetic knowledge.

Unsupervised Subtitle Segmentation with Masked Language Models (2023.acl-short)

Copied to clipboard

David Ponce, Thierry Etchegoyhen, Victor Ruiz

Improving LLM Domain Certification with Pretrained Guide Models (2026.eacl-long)

Challenge:	Existing methods to improve subtitle segmentation are based on character counting and linguistically correct segmentation.
Approach:	They propose a method where subtitle breaks are predicted according to likelihood of punctuation . their approach is highly portable across languages and domains .
Outcome:	The proposed method obtained competitive results in terms of segmentation accuracy across metrics while also fully preserving the original text and complying with length constraints.

Copied to clipboard

Jiaqian Zhang, Zhaozhi Qian, Faroq AL-Tam, Ignacio Iacobacci, Muhammad AL-Qurishi, Riad Souissi

Bottom-Up Synthesis of Knowledge-Grounded Task-Oriented Dialogues with Iteratively Self-Refined Prompts (2025.naacl-short)

Challenge:	Large language models (LLMs) generate off-domain or harmful responses when deployed in high-stakes domains.
Approach:	They propose a method that leverages pretrained language models as guide models to sharply distinguish acceptable from refused content.
Outcome:	The proposed approach exploits pretrained language models as guide models while aligned to the target domain.

Copied to clipboard

Kun Qian, Maximillian Chen, Siyan Li, Arpit Sharma, Zhou Yu

Natural Language Interfaces

Challenge:	Training conversational question-answering systems requires in-domain data, which is often scarce in practice.
Approach:	They propose a bottom-up approach where QA pairs are generated first and combined into a coherent dialogue.
Outcome:	The proposed approach produces more realistic and higher-quality dialogues compared to top-down methods.

Database Search Results Disambiguation for Task-Oriented Dialog Systems (2022.naacl-main)

Copied to clipboard

Kun Qian, Satwik Kottur, Ahmad Beirami, Shahin Shayandeh, Paul Crook, Alborz Geramifard, Zhou Yu, Chinnadhurai Sankar

Leveraging Generative Large Language Models with Visual Instruction and Demonstration Retrieval for Multimodal Sarcasm Detection (2024.naacl-long)

Challenge:	Task-oriented dialog systems can't handle multiplesearch results when querying a database due to the lack of such scenarios in existing datasets.
Approach:	They propose a task that focuses on disambiguating database search results by synthetically generating turns through a pre-defined grammar and collecting human paraphrases for a subset.
Outcome:	The proposed task improves performance on DSR-disambiguation even in the absence of in-domain data, suggesting it can be learned as a universal dialog skill.

Copied to clipboard

Binghao Tang, Boda Lin, Haolong Yan, Si Li

Multimodality

Challenge:	Existing methods for multimodal sarcasm detection do not fully utilize cross-modal features, limiting their performance on in-domain datasets.
Approach:	They propose a multimodal sarcasm detection model with a designed instruction template and a demonstration retrieval module.
Outcome:	The proposed model outperforms existing methods on in-domain datasets and achieves state-of-the-art performance.

Adversarial Domain Adaptation for Variational Neural Language Generation in Dialogue Systems (C18-1)

Copied to clipboard

Van-Khanh Tran, Le-Minh Nguyen

Divide, Conquer, and Combine: Mixture of Semantic-Independent Experts for Zero-Shot Dialogue State Tracking (2023.acl-long)

Challenge:	Existing methods to adapt to domains have shown promising results in how to reuse data in a domain-scalable framework efficiently.
Approach:	They propose an adversarial training procedure to train a Variational encoder-decoder based language generator via multiple adaptation steps.
Outcome:	The proposed method can adapt to a related domain using only a small amount of in-domain data.

Copied to clipboard

Qingyue Wang, Liang Ding, Yanan Cao, Yibing Zhan, Zheng Lin, Shi Wang, Dacheng Tao, Li Guo

Domain Knowledge Transferring for Pre-trained Language Model via Calibrated Activation Boundary Distillation (2022.acl-long)

Challenge:	Existing methods to enhance the zeroshot generalization of DST fail to effectively decouple semantics of samples, limiting the zero-shot performance of the system.
Approach:	They propose a new learning schema that explicitly disentangles the semantics of seen data and leverages the performance and robustness with the mixture-of-experts mechanism.
Outcome:	The proposed model achieves state-of-the-art on multiWOZ2.1 with 10M trainable parameters and is robust to the mixture-of experts mechanism.

Copied to clipboard

Dongha Choi, HongSeok Choi, Hyunju Lee

A Deep Generative Distance-Based Classifier for Out-of-Domain Detection with Mahalanobis Space (2020.coling-main)

Challenge:	Pretrained language models are used to boost their performance on downstream tasks . pretraining with in-domain texts requires considerable in- domain data and training resources .
Approach:	They propose a domain knowledge transferring framework for pre-trained language models without additional in-domain pretraining.
Outcome:	The proposed framework extracts domain knowledge from an existing in-domain pretrained language model and transfers it to other PLMs by applying knowledge distillation.

Copied to clipboard

Hong Xu, Keqing He, Yuanmeng Yan, Sihong Liu, Zijun Liu, Weiran Xu

Unsupervised Domain Adaptation for Neural Machine Translation with Domain-Aware Feature Embeddings (D19-1)

Challenge:	Existing methods for detecting out-of-domain (OOD) intents rely on manually labeled samples . a strong generative distance-based classifier can detect OOD samples in task-oriented dialog systems .
Approach:	They propose a generative distance-based classifier to detect out-of-domain (OOD) intents . they use Gaussian discriminant analysis to avoid over-confidence problems .
Outcome:	The proposed method outperforms baseline methods on four benchmark datasets.

Copied to clipboard

Zi-Yi Dou, Junjie Hu, Antonios Anastasopoulos, Graham Neubig

Cost-effective Selection of Pretraining Data: A Case Study of Pretraining BERT on Social Media (2020.findings-emnlp)

Challenge:	Recent studies have focused on domain adaptation for neural machine translation systems where in-domain data is scarce or nonexistent.
Approach:	They propose an approach that adapts models with domain-aware feature embeddings, which are learned via an auxiliary language modeling task.
Outcome:	The proposed model performs better in multiple experimental settings and with back translation.

Copied to clipboard

Xiang Dai, Sarvnaz Karimi, Ben Hachey, Cecile Paris

We Need To Talk About Random Splits (2021.eacl-main)

Challenge:	Recent studies show that domain-specific BERT models can be improved when in-domain data is used for pretraining.
Approach:	They propose to use Twitter and forum text as pretraining sources for two BERT models and use similarity measures to nominate in-domain data for pretraining.
Outcome:	The proposed method can be used to improve performance on downstream tasks by using in-domain data.

Copied to clipboard

Anders Søgaard, Sebastian Ebert, Jasmijn Bastings, Katja Filippova

Automatic Speech Recognition System-Independent Word Error Rate Estimation (2024.lrec-main)

Challenge:	argued that random splits, like standard splits lead to overly optimistic performance estimates.
Approach:	They argue that random splits, like standard splits lead to overly optimistic performance estimates.
Outcome:	The proposed method leads to more realistic performance estimates than standard splits.

Copied to clipboard

Chanho Park, Mingjie Chen, Thomas Hain

Multimodality

Challenge:	Word error rate (WER) is a metric used to evaluate the quality of transcriptions produced by Automatic Speech Recognition systems.
Approach:	They propose a hypothesis generation method for ASR system-dependent WER estimation . they use phonetically similar or linguistically more likely alternative words to generate hypotheses .
Outcome:	The proposed method outperforms baseline estimators on in-domain data and out-of-domain on Switchboard and CALLHOME.

Do Androids Know They’re Only Dreaming of Electric Sheep? (2024.findings-acl)

Copied to clipboard

Sky CH-Wang, Benjamin Van Durme, Jason Eisner, Chris Kedzie

Reformulating Domain Adaptation of Large Language Models as Adapt-Retrieve-Revise: A Case Study on Chinese Legal Domain (2024.findings-acl)

Challenge:	Detecting hallucinations in grounded generation tasks is commonly framed as a textual entailment problem.
Approach:	They develop probes that are narrowly trained to predict hallucination in a transformer language model.
Outcome:	The probes can detect hallucinations at many transformer layers outperforming baselines and human annotators on two out of three generation tasks.

Copied to clipboard

Zhen Wan, Yating Zhang, Yexiang Wang, Fei Cheng, Sadao Kurohashi

Probing Commonsense Explanation in Dialogue Response Generation (2021.findings-emnlp)

Challenge:	Recent large language models like GPT-4 have demonstrated astonishing zero-shot capabilities in general domain tasks, but they often generate content with hallucinations in specific domains such as Chinese law.
Approach:	They propose a framework for adapting large language models (LLMs) to Chinese legal domains by reformulating generation as an adapt-retrieve-revise process.
Outcome:	The proposed framework outperforms existing models in the Chinese legal domain by +33.6 points in the zero-shot setting.

Copied to clipboard

Pei Zhou, Pegah Jandaghi, Hyundong Cho, Bill Yuchen Lin, Jay Pujara, Xiang Ren

Dialogue Systems & Conversational Agents

Challenge:	Currently, response generation (RG) models do not understand human communication intents.
Approach:	They propose to examine commonsense reasoning implicitly to determine whether RG models produce coherent responses in conversations.
Outcome:	The proposed probing settings show that RG models fail to capture the logical relations between commonsense explanations and responses and fine-tuning on in-domain data do not lead to understanding of CSR for RG.

Data Augmentation for Cross-Domain Named Entity Recognition (2021.emnlp-main)

Copied to clipboard

Shuguang Chen, Gustavo Aguilar, Leonardo Neves, Thamar Solorio

Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue StateTracking (2021.naacl-main)

Challenge:	Existing methods for named entity recognition focus on augmenting in-domain data in low-resource scenarios where annotated data is limited.
Approach:	They propose a neural architecture to transform data from high-resource to low-resourced domains by learning the patterns in the text that differentiate them.
Outcome:	The proposed approach improves on high-resource domain representations over high- and low-resourced domains.

Copied to clipboard

Zhaojiang Lin, Bing Liu, Seungwhan Moon, Paul Crook, Zhenpeng Zhou, Zhiguang Wang, Zhou Yu, Andrea Madotto, Eunjoon Cho, Rajen Subba

Dialogue Systems & Conversational Agents

Challenge:	Existing models for zero-shot cross-domain dialogue state tracking require in-domain data to model a new domain.
Approach:	They propose a slot descriptions enhanced generative approach for zero-shot cross-domain DST by encoding a dialogue context and a slots with a pre-trained encoder and generating slot value in auto-regressive manner.
Outcome:	The proposed model significantly improves state-of-the-art results in zero-shot cross-domain setting.

Zero-shot Cross-domain Dialogue State Tracking via Context-aware Auto-prompting and Instruction-following Contrastive Decoding (2024.emnlp-main)

Copied to clipboard

Xiaoyu Dong, Yujie Feng, Zexin Lu, Guangyuan Shi, Xiao-Ming Wu

Soft Alignment Objectives for Robust Adaptation of Language Generation (2023.acl-long)

Challenge:	Previous studies have implemented slot-based input improvements, such as schema-driven descriptions and question-answering formats, but still suffer from negative transfer for seen slots and inefficient transfer for unseen slots due to the significant source-target domain gap.
Approach:	They propose a framework that generates dynamic, context-aware slot queries to improve model transferability by penalizing deviations from the provided instructions.
Outcome:	Experiments on two datasets show that the proposed model performs better than existing models on the restaurant domain.

Copied to clipboard

Michal Štefánik, Marek Kadlcik, Petr Sojka

Improving the Generalizability of Depression Detection by Leveraging Clinical Questionnaires (2022.acl-long)

Challenge:	Domain adaptation is a common approach for generative language models, but it is notorious for over-specialization to the target domain, resulting in catastrophic forgetting.
Approach:	They propose to build training objectives on a semantic similarity of predicted tokens to the reference and avoid catastrophic forgetting of adaptation by preserving adaptation in-domain quality.
Outcome:	The proposed objectives mitigate catastrophic forgetting while preserving the adaptation in-domain quality while reducing computational costs.

Copied to clipboard

Thong Nguyen, Andrew Yates, Ayah Zirikly, Bart Desmet, Arman Cohan

Challenge:	Existing approaches to identify mental health conditions using social media are limited by the presence of symptoms described in a questionnaire used by clinicians.
Approach:	They propose to ground a model in PHQ9's symptoms to improve generalization . they also show that this approach can still perform competitively on in-domain data.
Outcome:	The proposed approach can perform competitively on in-domain data while improving generalizability and generalisability.

Zero-Shot Dialogue State Tracking via Cross-Task Transfer (2021.emnlp-main)

Copied to clipboard

Zhaojiang Lin, Bing Liu, Andrea Madotto, Seungwhan Moon, Zhenpeng Zhou, Paul Crook, Zhiguang Wang, Zhou Yu, Eunjoon Cho, Rajen Subba, Pascale Fung

Natural Language Interfaces

Challenge:	Existing approaches to training a dialogue state tracking model require extensive annotated dialogue data.
Approach:	They propose to transfer cross-task knowledge from general question answering corpora to QA model that can handle zero-shot DST.
Outcome:	The proposed model improves existing zero-shot and few-shot results on MultiWoz and shows better generalization ability in unseen domains.

Explainable Automated Fact-Checking for Public Health Claims (2020.emnlp-main)

Copied to clipboard

Neema Kotonya, Francesca Toni

A Bag of Tricks for Dialogue Summarization (2021.emnlp-main)

Challenge:	a few blind spots exist in the state-of-the-art in fact-checking for political claims.
Approach:	They propose to use a dataset of 11.8K claims to explain fact-check labels for claims . they define and evaluate three coherence properties of explanation quality with humans .
Outcome:	The proposed model can be trained on in-domain data and evaluates its coherence properties with humans and computationally.

Copied to clipboard

Muhammad Khalifa, Miguel Ballesteros, Kathleen McKeown

Summarization

Challenge:	Using a pretrained sequence-to-sequence language model, we explore speaker name substitution, negation scope highlighting, multi-task learning with relevant tasks, and pretraining on in-domain data.
Approach:	They propose a pretrained sequence-to-sequence language model that can handle different parts of dialogue belonging to multiple speakers and combine them to produce a coherent monologue summary.
Outcome:	The proposed techniques outperform baseline models on a dialogue summarization dataset.

Improving Low Compute Language Modeling with In-Domain Embedding Initialisation (2020.emnlp-main)

Copied to clipboard

Charles Welch, Rada Mihalcea, Jonathan K. Kummerfeld

Domain adapted machine translation: What does catastrophic forgetting forget and why? (2024.emnlp-main)

Challenge:	Existing approaches to train language models on in-domain data are limited.
Approach:	They propose to initialise and freeze in-domain embeddings to provide a useful representation of rare words in English . they find that the standard configuration is not optimal when rare words are present .
Outcome:	The proposed approach improves language modeling by providing a useful representation of rare words in English.

Copied to clipboard

Danielle Saunders, Steve DeNeefe

Machine Translation

Challenge:	Neural Machine Translation (NMT) models can be specialized by domain adaptation, often fine-tuning on a dataset of interest.
Approach:	They propose a novel approach to understanding catastrophic forgetting during NMT adaptation by investigating the relationship between the data and the in-domain vocabulary coverage.
Outcome:	The proposed model can be specialized by fine-tuning on a domain of interest, but can fail to achieve the predicted quality of the target domain.

gaHealth: An English–Irish Bilingual Corpus of Health Data (2022.lrec-1)

Copied to clipboard

Séamus Lankford, Haithem Afli, Órla Ní Loinsigh, Andy Way

Multilinguality

Challenge:	Existing models for low-resource languages often focus on creating the largest possible dataset for generic translation.
Approach:	They develop a dataset for the specific domain of health for a low-resource English to Irish language pair and compare it to other similar datasets.
Outcome:	The proposed model improved BLEU score by 22.2 points compared with top performing models from the LoResMT2021 Shared Task.

JCoLA: Japanese Corpus of Linguistic Acceptability (2024.lrec-main)

Copied to clipboard

Taiga Someya, Yushi Sugimoto, Yohei Oseki

What Makes a Good Order of Examples in In-Context Learning (2024.findings-acl)

Challenge:	Neural language models have exhibited outstanding performance in downstream tasks, yet there is limited understanding regarding the extent of their internalization of syntactic knowledge.
Approach:	They introduce a dataset that analyzes sentences annotated with binary acceptability judgments from linguistic textbooks and handbooks and splits them into in-domain and out-of-domain data.
Outcome:	The proposed datasets show that models can surpass human performance for in-domain data while no models can exceed human performance on out-of-domain datasets.

Copied to clipboard

Qi Guo, Leiyu Wang, Yidong Wang, Wei Ye, Shikun Zhang

Label Confidence Weighted Learning for Target-level Sentence Simplification (2024.emnlp-main)

Challenge:	Large language models (LLMs) demonstrate impressive few-shot learning capabilities via in-context learning (ICL).
Approach:	They propose to use unlabeled data to evaluate order performance . they propose to filter out subsets of orders with label fairness and select the most influential order for each test instance.
Outcome:	The proposed method is superior over strong baselines and validates generalizability across settings.

Copied to clipboard

Xin Ying Qiu, Jingshen Zhang

Text Generation

Challenge:	Existing methods for sentence simplification use label confidence weighting to generate pseudo-labeled sentences with varying proficiency levels.
Approach:	They propose a label confidence weighting scheme for multi-level sentence simplification that incorporates a weighting system into the training loss of the encoder-decoder model.
Outcome:	The proposed approach outperforms state-of-the-art confidence weighting methods on English grade-level simplification datasets.

IOPO: Empowering LLMs with Complex Instruction Following via Input-Output Preference Optimization (2025.acl-long)

Copied to clipboard

Xinghua Zhang, Haiyang Yu, Cheng Fu, Fei Huang, Yongbin Li

Challenge:	Existing algorithms to improve the ability of LLMs to follow complex instructions are lacking.
Approach:	They propose a benchmark to improve the ability to follow complex instructions by using a IOPO alignment method to take input and output preference into consideration.
Outcome:	The proposed algorithm shows 8.15%, 2.18% improvements on in-domain data and 5.91%, 2.83% on out-of-domain datasets compared to SFT and DPO respectively.

Task Oriented In-Domain Data Augmentation (2024.emnlp-main)

Copied to clipboard

Xiao Liang, Xinyu Hu, Simiao Zuo, Yeyun Gong, Qiang Lou, Yi Liu, Shao-Lun Huang, Jian Jiao

Numerical Reasoning

Challenge:	Existing methods for large language models suffer from two major issues: in-domain data are scarce compared with general domain-agnostic data.
Approach:	They propose a task-oriented in-domain data augmentation framework that uses in- domain data selection and task-orientated synthetic passage generation to adapt LLMs to two domains: advertisement and math.
Outcome:	The proposed framework improves LLM performance by 8% in the advertisement domain and 7.5% in the math domain.

Diversification Catalyzes Language Models’ Instruction Generalization To Unseen Semantics (2025.findings-acl)

Copied to clipboard

Dylan Zhang, Justin Wang, Francois Charton

Beyond Generation: Leveraging LLM Creativity to Overcome Label Bias in Classification (2025.findings-acl)

Challenge:	Instruction-tuned language models excel in knowledge, reasoning, and instruction-following . however, the factors enabling generalization to unseen instructions remain underexplored .
Approach:	They propose to model instruction-following as a computational process and design controlled experiments inspired by the Turing-complete Markov algorithm to disentangle its dynamics.
Outcome:	The proposed model outperforms scaling up data volumes in generalist models by combining in-domain and diverse out-of-domain tasks.

Copied to clipboard

Xiaoyue Wang, Xin Liu

Model-Based Ranking of Source Languages for Zero-Shot Cross-Lingual Transfer (2025.emnlp-main)

Challenge:	Existing methods to mitigate label bias by leveraging in-domain data are often unavailable in real-world scenarios.
Approach:	They propose a calibration method that generates synthetic in-domain data from a few in-context demonstrations and utilizes it for calibration.
Outcome:	The proposed method reduces label bias by leveraging in-domain data from demonstrations.

Copied to clipboard

Abteen Ebrahimi, Adam Wiemerslage, Katharina von der Wense